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yaim 1 (Previously presented); A method comprising: 
idenMymg, in response to a search query, first multimedia objects having 
an associated keyv^orcl that matches a keyword in the search query and second 
multimedia objects that^ve content features similar to those of the first 
multimedia objects; 

presenting the first and second Multimedia objects to a user, 
monitoring feedback from the user a^to which of the first and second 
multimedia objects are relevant to the search query; 

annotating one or more of the multimedia object^swhich are deemed 
relevant by the user, with the keyword- 



Claim 2 (Original); A method as recited in claim 1 , further comprising: 



lining associations between 
the associations TJeing weighted to indic( 
multimedia objects; and 

adjusting the weights ol tfte.associafii 



ords and the multimedia objects, 
te howVelevant the keywords are to the 



fased on the user's feedback. 




Claim 3 (Original): A method as recit&d^n claim 2, wherein the adjusting 
comprises increasing a weight of an association between the keyword and a 
particular multimedia object that is deemed relevant by the us* 
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Clanh>4 (Original): A method as 
comprises decreasing a weight of an 
particular multimedia object that is deeme( \ 




in claim 2, wherein the adjusting 
between the keyword and a 
it by the user. 



Claim ^(Original): A methodT^recited in claim 4, further comprising 
removing the keyword from the particular n.oTtitxtedia object in an event that the 
weight is less than a threshold value. 

3- 

Claim y6 (Original): A method as recited in claim 1, further comprising 
training how the first and second multimedia objects are identified based on the 
user's feedback. 



4' 



Claim '^(Original): A method as recited in claim 1, further comprising 
refining the search to identify additional multimedia objects that contain content 
features similar to those of the multimedia objects indicated by the user as being 
relevant. 

Claim^Original): A method as recited in claim 1, wherein the multimedia 
objects comprise one of digital images, video objects, and audio objects. 

ClainK*9^ [Original): A computer readable medium having computer- 
executable instructions that, when executed on a processor, perform the method as 
recited in claim 1. 
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Claim I©v(Original): A method comprising: 

iteratively rem^viQe multimedia objects from a database and monitoring 
feedback from a user as to wietii^r the multimedia objects are relevant to a 
keyword in a search query; and 

annotating the multimedia objects based on^ftie^ser*& feedback, with the 
keyword* 

Claim 14 (Original): A method as recited in claim ^0, wherein the 
retrieving comprises using content-based information retrieval to retrieve the 
multimedia objects. 

Claim yi (Original): A method as recited in claim wherein the 
retrieving comprises using both content-based information retrieval and semantic- 
based information retrieval to retrieve the multimedia objects. 

ID- 1 

Claim ^^(Original): A method as recited in claim yb 9 wherein the 
monitoring comprises monitoring both feature-based relevance feedback and 
semantic-based relevance feedback. 

Claim J^r (Original): A method as recited in claim J*>, wherein the 
annotating is hidden from the user. 
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J, adding the keyword to the 



15 (Original): A method / sT"reqted in claim 10, wherein the 
annotating comJ>rtses: 

in an event thabagparticular multimedia object is deemed relevant by the 
user and is not yet annotated>^ith the kc 
particular multimedia object; and 

in an event that the particular multimedfe^ject is deemed relevant by the 
user and is already annotated with the keyword, strengthening an association 
between the keyword and the particular multimedia object. 

Claim ^I^tOriginal): A method as recited in claim -KJf wherein the 
annotating comprises: 

in an event that a particular multimedia object is deemed irrelevant by the 
user and is already annotated with the keyword, weakening an association between 
the keyword and the particular multimedia object 

Claim XI (Original): A computer readable medium having computer- 
executable instructions that, when executed on a processor, perform the method as 

J 

recited in claim 4#T 

m 18 (Original): A method comprising: 
retrievingNa^ultimedia objects according to a content-based retrieval 
process; 

presenting the multimedia otyfeet§Jo a user; 

monitoring feedback from the user as^Ux^hich of the multimedia objects 
are relevant; and 
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annotating one or 
feedback, with a keyword, 



^m&re^Qf the multimedia objects based on the user's 



15 
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Claim^J^ (Original): A method as recited in claim wherein the 
monitoring comprises monitoring both feature-based relevance feedback and 
semantic-based relevance feedback. 

Claim 20 (Original): A method as recited in claim & wherein the 
annotating is hidden from the user. 

21 (Original): A method^ j&_cecited in claim 18, wherein the 
annotating comjfris^s: 

in an event that ^particular multir 
user and not yet annotated witti s th^keywoi|d> 
multimedia object; and 

in an event that the particular multime 
user and is already annotated with the keyword, strengthening an association 
between the keyword and the particular multimedia object. 

Claim ^22^(Original): A method as recited in claim wherein 
annotating comprises: 

in an event that a particular multimedia object is deemed irrelevant by the 
user and is already annotated with the keyword, weakening an association between 
the keyword and the particular multimedia object. 




t is deemed relevant by the 
[ing the keyword to the particular 




object is deemed relevant by the 
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Claim pi (Original): A method as recited in claim wherein the 
annotating comprises: 

in an event that a particular multimedia object is deemed irrelevant by the 
user and is already annotated with the keyword, removing the keyword from the 
particular multimedia object. 



Claim (Original): A computer readable medium having computer- 
executable instructions that, when executed on a processor, perform the method as 
recited in claim 



Claim 25 (Original): A method comprising: 

rtaining associations between keywords and multimedia objects, the 



associations b&ng weighted to in< 

multimedia objects; 

retrieving a set ot Oi\e or more 
monitoring feedback frbm the 

are relevant; and 

adjusting the weights of the 




relevant the keywords are to the 



cpa objects for presentation to a user; 
^erjjs^ to which of the multimedia objects 





ions based on the user's feedback* 



Claim 26 (Original): A method as recited in claim 25, wherein the 
retrieving comprises using content-based information retrieval to retrieve the 
multimedia objects. 
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Claim 27 (Original): A method as recited in claim 25, wherein the 
retrieving comprises using both content-based information retrieval and semantic- 
based information retrieval to retrieve the multimedia objects. 



Claim 28, (Original): A method 
monitoring composes capturing both 
semantic-based relevaitee feedback. 




ted in claim 25, wherein the 
ed relevance feedback and 



Claim 29 (Original);\ A method a!£ recited in claim 25, wherein the 
adjusting comprises increasing\the weights of the associations between the 
keywords and the multimedia objec&.that are deemed relevant by the user 

Claim 30 (Original): A method \s recited in claim 25, wherein the 
adjusting comprises decreasing the weights^of the associations between the 
keywords and the multimedia objects that are deeme^ irrelevant by the user. 

Claim 31 (Original): A computer readable medra^n having computer- 
executable instructions that, when executed on a processor, perform the method as 
recited in claim 25. 

Ct£im42 (Original): A system comprising: 

an informatiraTretqeval unit to retrieve multimedia objects from a database 
based on a search query; 

a relevance feedback unit to captureV4jser's feedback as to whether the 
multimedia objects are relevant to the search query; ant 



8 



US*lS46Sl.t aiush 



SI-60iUSUO}.<he 



PAGE 10/25 1 RCVD AT 2127/2004 3:19:40 PM [Eastern Standard Tone] ' SVR:USPT0-EFXRM1 * DH1S:8729305 ' CSID:509 323 8979 1 OUR 




FEB 27 2004 12:30 FR LEE - RHYES PLL 



509 323 8979 TO 17038729306 



^^3 



P. 11/25 



5 
6 
7 
8 
9 

to 

It 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



an annotation uitit^to annotate, with a keyword, the multimedia objects 
based on the user's feedback. 



3o 



Claim ^3 (Original): A system as recited in claims, wherein the search 
query comprises a keyword-based search query having one or more keywords. 



claims?, * 



Claim ^"(Original): A system as recited in claim^Z, wherein the search 
query comprises a content-based search query having one or more content features. 

ft- Ab 

Claim^35^ (Original): A system as recited in claim ^32f wherein the 
information retrieval unit employs both content-based information retrieval and 
semantic-based information retrieval. 

Claim (Original): A system as recited in claim 32; wherein the 
information retrieval unit comprises: 

a query handler to handle both keyword-based queries having one or more 
search keywords and content-based queries having one or more content features of 
a multimedia object; and 

a feature and semantic matcher to identify at least one of (1) first 
multimedia objects having keywords that match the search keywords from a 
keyword-based query, and (2) second multimedia objects having content features 
similar to the content features of a content-based query. 
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Claim yf (Original): A system as recited in claim 32, wherein the relevance 
feedback unit employs both feature-based relevance feedback and semantic-based 
relevance feedback- 

Claimj^Original): A system as recited in claim wherein: 
the search query comprises a keyword-based search query having at least 
one keyword; and 

in an event that a particular multimedia object is deemed relevant by the 
user and is not yet annotated with the keyword, the annotation unit adds the 
keyword to the particular multimedia object. 

Claim 39 (Original): A system as recited in claim 32, wherein: 
the search query comprises a keyword-based search query having at least 
one keyword^nd 

in an eveit that a particular multimedia object is deemed relevant by the 
user and is already anh^tated with the keyword, r^ie annotation unit strengthens an 
association between the keyword and the^artteula^lnultimedia object. 



ecited in claim 32, wherein: 
itord-based search query having at least 



Claim 40 (Original): Asysten^as 
the search query comprises a ke; 
one keyword; and 

in an event that a particular multimedia objfe^t is deemed irrelevant by the 
user and is already annotated with the keyword, weakening an association between 
the keyword and the particular multimedia object 
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Claim AY 

the search query" 
one keyword; and 

in an event that a particular imi 
user and is already annotated with the 
particular multimedia object. 



in claim 32, wherein: 
ed search query having at least 



object is deemed irrelevant by the 
removing the keyword from the 



Claim>l^(Original): An image retrieval system as recited in claim^T 
wherein the relevance feedback unit comprises a feedback analyzer to train the 
system based on the user's feedback. 



Claims 43-44 (Previously withdrawn) 
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